The following report contains model performance metrics for the NY City Hourly Probabilistic Residential Energy Demand Forecasting Pipeline.
Model performance was evaluated on both long-term and day-ahead forecasts. Evaluation was conducted using a holdout dataset of hourly energy
demand values between 2023-09-26 and 2024-04-27.
Long-Term Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
128742.60 |
126301.35 |
270.23 |
0.05 |
0.0 |
| Baseline Yearly MA |
604819.81 |
604819.81 |
626.46 |
0.13 |
NaN |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
Day-Ahead Forecasting Performance
The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
|
MSE |
Weighted MSE |
MAE |
MAPE |
Wilcoxon Test p-value |
| Forecasting Pipeline |
121216.29 |
121213.86 |
265.82 |
0.05 |
NaN |
| Baseline Moving Avg |
353146.56 |
353146.56 |
515.37 |
0.10 |
0.0 |
| EIA Forecasts |
113145.06 |
NaN |
291.30 |
0.06 |
0.0 |
The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand
values. It also presents the 95% confidence interval bounds estimated by the forecasting model.
The following data has been made up for demonstration purposes. Actual data will be provided in a future update.